Global Data Quality Assessment and the Situated Nature of "Best" Research Practices in Biology

This is the author accepted manuscript. The final version is available from Ubiquity Press via the DOI in this record.This paper reflects on the relation between international debates around data quality assessment and the diversity characterising research practices, goals and environments within the life sciences. Since the emergence of molecular approaches, many biologists have focused their research, and related methods and instruments for data production, on the study of genes and genomes. While this trend is now shifting, prominent institutions and companies with stakes in molecular biology continue to set standards for what counts as ‘good science’ worldwide, resulting in the use of specific data production technologies as proxy for assessing data quality. This is problematic considering (1) the variability in research cultures, goals and the very characteristics of biological systems, which can give rise to countless different approaches to knowledge production; and (2) the existence of research environments that produce high-quality, significant datasets despite not availing themselves of the latest technologies. Ethnographic research carried out in such environments evidences a widespread fear among researchers that providing extensive information about their experimental setup will affect the perceived quality of their data, making their findings vulnerable to criticisms by better-resourced peers. These fears can make scientists resistant to sharing data or describing their provenance. To counter this, debates around Open Data need to include critical reflection on how data quality is evaluated, and the extent to which that evaluation requires a localised assessment of the needs, means and goals of each research environment.This research was funded by the European Research Council grant award 335925 (“The Epistemology of Data Science”), the Leverhulme Trust Grant number RPG-2013-153 (“Beyond the Digital Divide”), and the Australian Research Council, Discovery Project DP160102989 (“Organisms and Us”). Th

Open Data: curation Is under-resourced

The final published version is available from the publisher via the DOI in this record.Correspondenc

Incentives and Rewards to Engage in Open Science Activities

This is the final version of the report. Available from the European Commission via the link in this record.This report has been produced following the 3rd working meeting of the participants in the Mutual Learning Exercise (MLE) on Open Science, which was hosted by Croatia in Dubrovnik on 12 and 13 September 2017. It provides an overview and assessment of the various practices currently being used and/or investigated to incentivise and reward researchers and their institutions for engaging in open science activities. The report starts with a section (section 2) outlining the Open Science agenda and aims and its role within the broader research and science policy landscape. Section 3 outlines the advantages and challenges underpinning the implementation of Open Science, thereby providing the necessary background to the discussion on incentives and rewards which can foster such activities. Section 4 reports on the discussions emerging from the MLE participants and outlines key concerns and feedback gathered by Member States on how Open Science can and should be fostered. Sections 4, 5 and 6 detail the incentives and rewards that could be provided, or in some cases have already been implemented, by three groups of key stakeholders: researchers themselves; research-performing institutions and funding bodies; and national governments. In conclusion, a summary is made of the main advantages and disadvantages of each type of incentive, with suggestions as to who is mainly responsible for managing its implementation. The report is based on a review of relevant background academic literature and policy documents, discussions at previous MLE meetings (particularly the one on alternative metrics for Open Science, which took place in May 2017 in Helsinki), and on answers to open-ended questions sent to the MLE participants ahead of the meeting. Data have also been sourced from the European Open Science Monitor which, at the time of writing, is the most comprehensive source of information on Open Science implementation policies across European Member States (http://ec.europa.eu/research/openscience).European Commissio

Locating Ethics in Data Science: Responsibility and Accountability in Global and Distributed Knowledge Production

This is the author accepted manuscript. The final version is available from Royal Society via the DOI in this record.The distributed and global nature of data science creates challenges for evaluating the quality, import and potential impact of the data and knowledge claims being produced. This has significant consequences for the management and oversight of responsibilities and accountabilities in data science. In particular, it makes it difficult to determine who is responsible for what output, and how such responsibilities relate to each other; what ‘participation’ means and which accountabilities it involves, with regards to data ownership, donation and sharing as well as data analysis, re-use and authorship; and whether the trust placed on automated tools for data mining and interpretation is warranted (especially since data processing strategies and tools are often developed separately from the situations of data use where ethical concerns typically emerge). To address these challenges, this paper advocates a participative, reflexive management of data practices. Regulatory structures should encourage data scientists to examine the historical lineages and ethical implications of their work at regular intervals. They should also foster awareness of the multitude of skills and perspectives involved in data science, highlighting how each perspective is partial and in need of confrontation with others. This approach has the potential to improve not only the ethical oversight for data science initiatives, but also the quality and reliability of research outputs.This research was funded by the European Research Council grant award 335925 (“The Epistemology of Data-Intensive Science”), the Leverhulme Trust Grant number RPG-2013- 153 and the Australian Research Council, Discovery Project DP160102989

The Time of Data: Time-Scales of Data Use in the Life Sciences

This is the author accepted manuscript. The final version is available from University of Chicago Press via the DOI in this record.This paper considers the temporal dimension of data processing and use, and the ways in which it affects the production and interpretation of knowledge claims. I start by distinguishing the time at which data collection, dissemination and analysis occur (Data time, or Dt) from the time in which the phenomena for which data serve as evidence operate (Phenomena time, or Pt). Building on the analysis of two examples of data re-use from modelling and experimental practices in biology, I then argue that Dt affects how researchers (1) select and interpret data as evidence and (2) identify and understand phenomena.This research was funded by the European Research Council under the European Union's Seventh Framework Programme (FP7/2007-2013) / ERC grant agreement n° 335925 (project “The Epistemology of Data-Intensive Science”); and the ARC Discovery Grant “Organisms and Us” (DP160102989)

Re-Thinking Reproducibility as a Criterion for Research Quality

This is the author accepted manuscript. The final version is available from Emerald via the DOI in this record.A heated debate surrounds the significance of reproducibility as an indicator for research quality and reliability, with many commentators linking a "crisis of reproducibility" to the rise of fraudulent, careless and unreliable practices of knowledge production. Through the analysis of discourse and practices across research fields, I point out that reproducibility is not only interpreted in different ways, but also serves a variety of epistemic functions depending on the research at hand. Given such variation, I argue that the uncritical pursuit of reproducibility as an overarching epistemic value is misleading and potentially damaging to scientific advancement. Requirements for reproducibility, however they are interpreted, are one of many available means to secure reliable research outcomes. Furthermore, there are cases where the focus on enhancing reproducibility turns out not to foster high-quality research. Scientific communities and Open Science advocates should learn from inferential reasoning from irreproducible data, and promote incentives for all researchers to explicitly and publicly discuss (1) their methodological commitments, (2) the ways in which they learn from mistakes and problems in everyday practice, and (3) the strategies they use to choose which research component of any project needs to be preserved in the long term, and how.This research was funded by the European Research Council grant award 335925 (“The Epistemology of Data-Intensive Science”), the Australian Research Council Discovery Project “Organisms and Us” and the UK Economic and Social Research Council award ES/P011489/1

Data Governance is key to interpretation: reconceptualizing data in data science

This is the author accepted manuscript. The final version is available from MIT press via the DOI in this record.I provide a philosophical perspective on the characteristics of data-centric research and the conceptualization of data that underpins it. The transformative features of contemporary data science derive not only from the availability of Big Data and powerful computing, but also from a fundamental shift in the conceptualization of data as research materials and sources of evidence. A relational view of data is proposed, within which the meaning assigned to data depends on the motivations and instruments used to analyze them and to defend specific interpretations. The presentation of data, the way they are identified, selected and included (or excluded) in databases and the information provided to users to re-contextualize them are fundamental to producing knowledge - and significantly influence its content. Concerns around interpreting data and assessing their quality can be tackled by cultivating governance strategies around how data are collected, managed and processed.European CommissionAustralian Research CouncilAlan Turing Institut

What Distinguishes Data from Models?

This is the final version. Available on open access from Springer Verlag via the DOI in this recordI propose a framework that explicates and distinguishes the epistemic roles of data and models within empirical inquiry through consideration of their use in scientific practice. After arguing that Suppes’ characterization of data models falls short in this respect, I discuss a case of data processing within exploratory research in plant phenotyping and use it to highlight the difference between practices aimed to make data usable as evidence and practices aimed to use data to represent a specific phenomenon. I then argue that whether a set of objects functions as data or models does not depend on intrinsic differences in their physical properties, level of abstraction or the degree of human intervention involved in generating them, but rather on their distinctive roles towards identifying and characterizing the targets of investigation. The paper thus proposes a characterization of data models that builds on Suppes’ attention to data practices, without however needing to posit a fixed hierarchy of data and models or a highly exclusionary definition of data models as statistical constructs.European CommissionAustralian Research Counci

Scientific Agency and Social Scaffolding in Contemporary Data-Intensive Biology

This is the author accepted manuscript. The final version is available from University of Minnesota Press via the link in this recordIt is widely recognised that social scaffolding is crucial to the entrenchment of new technologies and related standards and practices in scientific research, as well as to its manifestations and results. At the same time, there is little understanding of the circumstances under which, and the reasons why, some forms of sociality are effective in promoting particular types of scientific work. This chapter explores these questions by focusing on two forms of social scaffolding involved in the development of practices of data dissemination through digital means – and particularly infrastructures such as online databases – within the contemporary life sciences: (1) ontology consortia, which have recently emerged as de facto regulatory bodies for data curation in the US and Europe, and (2) steering committees for model organism communities, which play significant roles in the governance of biological research in the UK. I discuss the successful transformation of these initially ad hoc groups into scientific institutions with political and epistemic visibility and power. Drawing on political theory, I then argue that viewing these organisations as social movements is a fruitful strategy to understand their development from informal gatherings into well-recognised regulatory bodies, and how this process of institutionalisation builds on highly entrenched forms of group socialisation. This in turn facilitates an analysis of the interrelation between institutional and infrastructural scaffolding involved in the evolution of scientific knowledge-making activities.This research was funded by the European Research Council grant award 335925

How Does One “Open” Science? Questions of Value in Biological Research

Open Science policies encourage researchers to disclose a wide range of outputs from their work, thus codifying openness as a specific set of research practices and guidelines, which can be interpreted and applied consistently across disciplines and geographical settings. In this paper, we argue that this “one-size-fits-all” view of openness sidesteps key questions about the forms, implications, and goals of openness for research practice. We propose instead to interpret openness as a dynamic and highly situated mode of valuing the research process and its outputs, which encompasses economic as well as scientific, cultural, political, ethical and social considerations. This interpretation sets up a critical space for moving beyond the economic definitions of value embedded in the contemporary biosciences landscape and Open Science policies, and stress the diversity of interests and commitments that affect research practices in the life sciences. To illustrate these claims, we use three case studies that highlight the challenges surrounding decisions about how – and how best – to make things open. These cases, which are drawn from interviews carried out with UK-based biologists and bioinformaticians in 2013 and 2014, show how the enactment of openness reveals judgments about what constitutes a legitimate intellectual contribution, for whom, and with what implications